Apparatus and method for proximity based input

ABSTRACT

A method is provided for controlling proximity detection based user interface functions. In response to detection of simultaneous presence of a first object and a second object in close proximity to the input surface, an input mode may be activated, in which input mode a first action is associated with an input by the first object and a second action is associated with an input by the second object.

TECHNOLOGICAL FIELD

Embodiments of the present invention relate to an apparatus and a methodfor detecting proximity based input.

BACKGROUND

Touch screens are used in many portable electronic devices, for instancein PDA (Personal Digital Assistant) devices, tabletops, and mobiledevices. Touch screens are operable by a pointing device (or stylus)and/or by finger. Typically the devices also comprise conventionalbuttons for certain operations.

Three dimensional (3D) content display techniques and 3D interaction, orsense of 3D interaction, have been under development also for portableelectronic devices. It has been suggested to apply also the space abovea touch screen surface as further means for user inputs.

SUMMARY

Various aspects of examples of the invention are set out in the claims.

According to an aspect, an apparatus is provided, comprising at leastone processor; and at least one memory including computer program code,the at least one memory and the computer program code configured to,with the at least one processor, cause the apparatus at least toperform: detect presence of a first object in close proximity to aninput surface, detect presence of a second object in close proximity tothe input surface, and in response to the detection of simultaneouspresence of the first object and the second object, activate an inputmode in which a first action is associated with an input by the firstobject and a second action is associated with an input by the secondobject.

According to an aspect, a method is provided, comprising: detectingpresence of a first object in close proximity to an input surface,detecting presence of a second object in close proximity to the inputsurface, and in response to the detection of simultaneous presence ofthe first object and the second object, activating an input mode inwhich a first action is associated with an input by the first object anda second action is associated with an input by the second object.

According to an aspect, there is provided a user interface for anelectronic device comprising a proximity detection system with at leastone proximity detector for detecting presence of a first object and asecond object in close proximity to an input surface, wherein the userinterface is configured to operate in an input mode, in which a firstaction is associated with an input by the first object and a secondaction is associated with an input by the second object, in response tothe detection of simultaneous presence of the first object and thesecond object.

According to an embodiment, the input mode is deactivated in response todetecting removal of the first object and/or the second object fromclose proximity to the input surface.

The various embodiments of the invention provide several advantages,which will become apparent from the detailed description below.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of example embodiments of the presentinvention, reference is now made to the following descriptions taken inconnection with the accompanying drawings in which:

FIG. 1 is a simplified block diagram of a side view of an inputapparatus in accordance with an embodiment of the invention;

FIG. 2 is a simplified block diagram of a side view of an apparatus inaccordance with an embodiment of the invention,

FIG. 3 illustrates a method according to an embodiment of the invention;

FIG. 4 illustrates a method according to an embodiment of the invention;

FIGS. 5a and 5b show an example of a method according to an embodimentof the invention;

FIGS. 6a to 6d illustrate a sequence of user interface actions accordingto an embodiment of the invention;

FIG. 7 illustrates a top view of a display with a cursor in relationwith an object according to an embodiment of the invention; and

FIG. 8 illustrates an electronic device in accordance with an embodimentof the invention.

DETAILED DESCRIPTION

FIG. 1 illustrates an apparatus 10 with one or more input and/or outputdevices. The input devices may for example be selected from buttons,switches, sliders, keys or keypads, navigation pads, touch pads, touchscreens, and the like. The output devices may be selected from displays,speakers, indicators, for example.

The apparatus 10 further comprises a proximity detection system or unit120 configured to detect when an object 110, 112, such as a finger or astylus, is brought in close proximity to, but not in contact with, aninput surface 100. The input surface 100 may be a surface of a touchscreen or other input device of the apparatus capable of detecting userinputs. For instance, the input surface 100 could be provided at housingin connection with an input device, such as a button, or as a specificinput area on side(s) or back (in view of the position of a display) ofa handheld electronic device.

A sensing area 140 may illustrate the approximate area and/or distanceat which an object 110, 112 is detected to be in close proximity to thesurface 100. The sensing area 140 may also be referred to as a hoveringarea and introduction of an object 110, 112 to the hovering area andpossible further (non-touch) inputs by the object 110, 112 in thehovering area may be referred to as hovering. In some embodiments thehovering area 140 enables also inputting and/or accessing data in theapparatus 10, even without touching the input surface 100.

The apparatus 10 further comprises or is coupled to a controller 130.The proximity detection system 120 is configured to provide thecontroller 130 with signals when an object 110, 112 enters the hoveringarea 140. Based on such input signals, commands, selections and othertypes of actions may be initiated, typically causing visible, audibleand/or tactile feedback for a user.

The apparatus 10 may be a peripheral device, such as a keyboard ormouse, or integrated in an electronic device. Examples of electronicdevices include any consumer electronics device like computers, mediaplayers, wireless communications terminal devices, and so forth.

In some embodiments, a proximity detection system 120 is provided in anapparatus comprising a touch screen display. FIG. 2 illustrates anapparatus 200 according to an embodiment comprising a touch screen 210.A plurality of touch sensitive detectors 212 is provided to detectinputs to screen display surface portions.

The apparatus 200 comprises a proximity detection circuit 220 fordetecting objects 110, 112 in close proximity to the surface of thetouch screen 210. In some embodiments the detection system 220 generatesa sensing field by one or more proximity sensors 222.

In one embodiment a capacitive proximity detection system is applied,whereby the sensors 222 are capacitive sensing nodes. Disturbances byone or more objects 110, 112 in the sensing field are monitored andpresence of one or more objects is detected based on detecteddisturbances. A capacitive detection circuit 220 detects changes incapacitance above the surface of the touch screen 210.

However, it will be appreciated that the present features are notlimited to application of any particular type of proximity detection.The proximity detection system 120, 220 may be based on infraredproximity detection, optical shadow detection, acoustic emissiondetection, ultrasonic detection, or any other suitable proximitydetection technique. For instance, in case the proximity detectionsystem 120, 220 would be based on infrared detection, the system wouldcomprise one or more emitters sending out pulses of infrared light. Oneor more detectors would be provided for detecting reflections of thatlight from nearby objects 110, 112. If the system detects reflectedlight, then an object is assumed to be present. The detection system120, 220 may be arranged to estimate the distance of an object from theinput surface. Depending on the technique applied, the size of theapparatus and the input surface, and the desired user interaction, thehovering area 140, 240 may be arranged to extend from the input surfaceby distance selected from some millimeters to even up to multiple dozensof centimeters, for instance. The proximity detection system 120, 220may detect also further parts of user's hand, but the system may bearranged to recognize such false inputs and avoid further actions.

The detection circuit 220 is coupled to a controller 230. Inputs to thetouch screen detectors 212 may be signalled via a control circuitry tothe controller 230, or another controller. The controller 230 may alsobe connected to one or more output devices, such as the touch screendisplay 210. The controller 230 may be configured to control differentapplication views on the display 210.

The proximity detection system 120, 220 is configured to detect andindicate simultaneous proximity of multiple objects 110, 112. Each ofthe objects 110, 112 detected simultaneously in the hovering area 140,240 may be associated with at least one action or functionality, whichis initiated in response to a further input by the respective object.

Further inputs may be provided by the objects 110, 112 in the hoveringarea 240, within respective input areas 250, 252. The apparatus 200 maybe configured to detect such further inputs in response to detecting oneor more of the objects 110, 112 pressing the touch screen displaysurface and/or performing a specific gesture in the hovering area 240. Abroad range of functions is available for selection to be associatedwith an input detected by a touch sensitive detection system and/or theproximity detection system 220. The controller 230 may be configured toadapt the associations according to a current operating state of theapparatus 200, a user input or an application executed in the apparatus200, for instance. For instance, associations may be applicationspecific, menu specific, view specific and/or context (which may bedefined on the basis of information obtained from the currentenvironment or usage of the apparatus 200) specific. It is to be notedthat one or more input areas 250, 252 and/or objects 110, 112 may be setto represent a particular action regardless of the current operatingstate of the electronic device. One or more associations may be adaptedspecifically in response to detecting that two or more objects 110, 112are introduced in the hovering area 140, 240.

FIG. 3 shows a method for controlling a multi-object hovering input modeaccording to an embodiment. The method may be applied as a controlalgorithm by the controller 130, 230, for instance. The method starts instep 302, whereby simultaneous presence of two or more objects in closeproximity to an input surface 210 is detected. Thus, step 302 may beentered when the two or more objects 110, 112 have been detected in thehovering area 140, 240, even though physical contact is not made withthe input surface. There may be further conditions, such as that theobjects are within a predefined distance from each other, or that one ormore of the objects are within a predefined distance from an object orsurface of an input surface of an electronic device.

In response to the detection of step 302, user interface functionalityis adapted by initiating 304 a specific input mode, herein referred toas a multi-object hovering input mode. Thus, one or more specificactions may be associated or linked with the first object and/or thesecond object, which actions are not available during separate use ofthe first or second object.

The method illustrated in FIG. 3 continuously checks 306 for hoveringand/or touch-based user inputs. In response to initiating themulti-object hovering mode, the method may start to determine the sourceof an input. If an input by the first object is detected 308, an actionassociated with the first object may be performed 310. If in step 312 aninput by the second object is detected, an action associated with thesecond object may be performed 314.

Although not illustrated in FIG. 3, the method typically also checks thetarget (information element) of the input, for instance an applicationarea or window or a particular graphical user interface (GUI) element onthe display. Thus, a software function currently associated with ahovering input area 250, 252 may be determined.

The method further checks 316 if removal of one or both objects fromclose proximity to the surface 100 has been detected. If yes, in step318 the multi-object hovering mode is deactivated or ended. Otherwise,the method may return to step 306 to check for subsequent user inputs.Although not illustrated in FIG. 3, the procedure may return fromoperations 310 and 314 to operation 306 to check for further user inputsor to operation 316.

The source of the input, e.g., whether an input is provided by the firstor second object (or a further object) may be determined 306, 308, 312in various ways. In one embodiment the proximity detection system 120,220 and/or controller 130, 230 is arranged to determine the source ofthe input on the basis of analyzing the position of the detected inputin relation to the other object's sensed position. On the basis of thedetected relational position, the object providing the input may bedetermined 308, 312 and associated action selected 310, 314. Theprocedure may determine the position of the inputting object in relationto the x-axis and/or y-axis (along the input surface). For instance, ifthe input by an object 112 is detected on left side of the other(hovering) object 110 is detected. The position of the objects inrelation to each other may be determined in response to detecting aninput or monitored continuously. In another embodiment, the proceduremay be arranged to determine the source of the input on the basis of oneor more sensed properties of the objects 110, 112. Instead of inaddition to utilizing sensed information on (ends) of the objects in thehovering area, further detected information e.g. on user's hand may beutilized when determining the inputting object. For instance, theapparatus 10, 200 may be arranged to detect the user's hand and if leftor right hand is used for hovering.

FIG. 4 illustrates a method according to another embodiment. The methodmay be applied as a control algorithm in the controller 130, 230checking detected inputs and controlling further user interfacefunctions based on detected inputs.

In step 402 presence of an object is detected in close proximity to afirst input area. For instance, the object 110 of FIG. 2 is detected tobe in close proximity to the first input area 250. An action associatedwith the first input area is initiated 404. For instance, a graphicaluser interface (GUI) element may be displayed nearby the first inputarea, but a wide variety of other action options also exist.

The method continues to monitor any hovering inputs. The simplified FIG.4 illustrates the detection of presence of a second object in closeproximity to a second input area in step 406. For instance, the object112 may be detected in the hovering area above the second input area 252of the touch screen display 210.

The method may in step 408 check if the first object still remains inclose proximity to the input surface 100, in the example of FIG. 2 inthe hovering area 240 or the specific input area 250. If so, in step 410user interface functionality associated with the first input area and/orthe second input area is adapted. Thus, the multi-object hovering modeillustrated in connection with FIG. 3 and step 304 may be initiated.

If the first object has been removed from the hovering area 240, themulti-object hovering mode may be deactivated, as illustrated inconnection with step 318, and an action associated with the second inputarea in a single-input hovering mode may be initiated 412.

The present features enable to bring further versatility and inputoptions for user interfaces with hovering capability. In case ofapplication of two objects, a third input level or configuration is thusprovided by the simultaneous presence of the two objects at the hoveringarea. Predetermined object-specific actions during the multi-objectinput mode may be provided, even regardless of the position of the inputin relation to the input area.

In some embodiments the multi-object hovering mode provides at leastsome functions similar to the functions of a computer mouse. FIG. 5aillustrates an embodiment in which functionality of left mouse button isassociated with the first (or second) object during the presence of atleast two objects in close proximity to the input surface 100.

In step 502 an input associated with information element is detected.For instance, an input by the first object 110 to the input area 240displaying a GUI element is detected. A selection action is performed504 for the information element. The information element may bedisplayed as selected, or a further action may be performed for theselected information element. For instance, in case of double-click typeof double input by the first object to an item in a list of items causesfirst selection of the item and then opening of a window for theselected item.

FIG. 5b illustrates basic features for a typical right mouse buttoninput. In step 552 an input associated with an information element isdetected. For instance, a hovering input or an input to the touch screen210 surface by the second object 112 to a GUI element displayed on thesecond area 242 is detected. In step 554 a menu of available actions isdisplayed. The menu may be context-sensitive and dependent on currentstate of the user interface or an application, application window orinformation element being selected, for instance. The menu may includetextual information, images and/or any other graphical information.

Thus, dedicated mouse button functionality may be enabled by introducingthe at least two objects in the hovering area 140, 240. The right orleft mouse action may be selected by the further input with theassociated object.

FIGS. 6a to 6d illustrate a user interface interaction sequenceaccording to an embodiment. FIG. 6a illustrates the initial displaycontents with a plurality of image GUI elements. When a user moves twoof his fingers on top of the screen and in close proximity to the screensurface, for instance the index and middle finger 110, 100 asillustrated in FIG. 6b , a proximity detection system detects that nowthere are two fingers on top of the screen area. A control system, forinstance the controller 130, 230 initiates the multi-object hoveringmode by linking functionality of left mouse button to the left fingerand right mouse button functionality to the right finger. If the userpresses the screen with one of the fingers, then the press activateseither the right or left mouse button functionality depending on whichfinger was used.

As illustrated in FIG. 6c , if the user presses the image1 602 with hisleft finger such detected input is associated with functionality,similar to the functionality associated with pressing a left mousebutton, and an image associated with the input area may be opened 610 tofull screen mode.

As illustrated in FIG. 6d , if the user presses the image1 602 with hisright finger 100, such detected input is associated with functionalitysimilar to right mouse button functionality and a pop-up menu 620 may beactivated.

The above-illustrated functionality enables to provide a virtualhovering mouse, which can be used across the UI as a replacement fortraditional mouse functionality. Conventional mouse click functionalityand logic may be provided by the hovering mouse. In addition or insteadof the above-illustrated functions, further mouse button functionality,including non-click interaction such as scrolling, and variouscombinations of two or more mouse buttons may be applied to offerfurther and new UI interactions.

In one embodiment the proximity detection system may be arranged todetect combined use of two or more objects, which may be interpreted asa third type of input in the multi-object hovering input mode. Anoperation may be associated with the combination of inputs by two ormore objects 110, 112. For instance, a multi-touch input simultaneouslywith two objects may be associated with another action than that of atouch input by only a single object. Thus, there may be one or morefurther checks for inputs in addition to the checks 410, 412, such as athird check for simultaneous input by the first and the second object.

The actions illustrated in connection with FIGS. 5a to 6d typicallyassociated with mouse buttons represent only some examples of availableactions which may be associated with the objects 110, 112 during themulti-object hovering mode. For instance, such actions may be selectedfrom the group of panning, rotating, scrolling, editing, zooming,reshaping of objects, etc.

FIGS. 3 to 6 d above illustrate only some of the available methods andfunctionality to arrange a multi-object functionality in response todetecting two or more objects in close proximity to an input surface.Some further or alternative embodiments are illustrated below.

According to some embodiments, more than two objects 110, 112 aresimultaneously used in the hovering area 140, 240, and specificfunctions may be triggered in response to detecting further objects. Forinstance, introduction of a combination of three or four fingers in thehovering area 140, 240 may trigger a further specific functionality ormode. One further example is that three object virtual mousefunctionality is initiated in response to detecting three fingers in thehovering area. The middle object may be associated with rollfunctionality, for instance.

According to some embodiments, at least one association to a graphicaluser interface element to be displayed is modified in response toactivation 304, 410 of and/or an action during the multi-object hoveringmode. This generally refers to adding and/or modifying one or moreinformation elements on the display. For instance, a specific GUI isdisplayed temporarily or during the multi-object hovering mode. Furtheroutputs, e.g. an audible and/or tactile output may also be produced bythe apparatus 10, 200 in response to entering the multi-object hoveringmode and/or in response to detection of an input during the multi-objecthovering mode. Thus, the user may be informed that the specificmulti-object hovering mode is supported and the input logic is changed.In a further embodiment an area of the touch screen is expanded duringthe multi-object hovering mode. An area may be expanded also in responseto detecting an object over the area during the multi-object hoveringmode.

Software functions or actions available during the multi-object hoveringmode and associated with the hovering area 140, 240, and/or specificinput area 250, 252 may be indicated to the user, for instance on thetouch screen 210. The function may be indicated when the function isavailable in the respective area 140, 240, 250, 252 and/or when an inputto the hovering area 250, 252 has been detected.

In one embodiment the object(s) detected in the hovering area 140, 240and/or the actions available by the respective object(s) are indicatedto the user during the multi-object hovering mode. For instance, eachavailable action is indicated by a GUI element associated with a GUIvisualizing the respective object 110, 112. Visualizations of theactions and/or objects may be displayed on a screen, and in case ofautostereoscopic displays also various 3D visualization options exist.

In one embodiment the objects 110, 112 detected in the hovering area140, 240 are displayed. For instance, an image of one, two or morefingers is displayed in accordance with the number of objects detectedin the hovering area 140, 240. In another example, in connection toapplying the visual mouse mode, an image of a mouse may be displayed. Ina still further example the apparatus 10, 200 may be arranged to displaynon-transparent or semi-transparent shadows of the objects 110, 112 inthe hovering area 140, 240. Furthermore, actions detected 308, 312 forthe objects may be indicated by changing the visualization of thedisplayed objects. For instance, the left-side button of the displayedmouse is moved back in response to detecting the input by the firstobject 110. Further, an action available by a further input by an objectmay be indicated in connection with the image or shadow of therespective object.

In one embodiment the apparatus 10, 200 is configured to cause displayof a pointer element in response to the detection of the presence of thefirst object and the second object 110, 112 in close proximity to theinput surface 100. The simplified FIG. 7 illustrates a pointer 700displayed on a touch screen. The displayed position of the pointerelement 700 is changed in relation to detected movement of the firstobject and/or the second object. Thus, the user may easily detect that amulti-object hovering mode with a virtual mouse type of functionality isactive.

In a further embodiment the pointer 700 indicates the target screen areafor input made during the multi-object hovering mode. Thus, furthervirtual mouse logic may be provided by the user interface, and itbecomes easier for a user to provide inputs to GUI elements that aresmall compared to the hovering area 140, 240. For instance, featuresrelated to displaying a cursor described in patent application No.PCT/IB2010/050752, incorporated herein by reference, may be applied inconnection with the present multi-object hovering input mode.

In one embodiment the multi-object hovering input mode applies only tosome of the items displayed on the screen of the apparatus 10, 200. Thecontroller 130, 230 may be arranged to inform the user of which UIobjects support the multi-object hovering input mode, such as thevirtual mouse button functionality illustrated in connection with FIGS.5a to 6d , in response to activation of and/or during the multi-objecthovering input mode. For instance, such UI objects may be enlarged,brought in front in case of 3D visualization, moved closer to theobjects, or the visualization of these items may be changed.

The apparatus 10, 200 may be configured to determine distance betweenthe objects 110, 112 in the hovering area 140, 240. Actions may beselected and during the multi-input hovering mode in accordance with thedetected distance (or change of distance). For instance, in response todetecting the first object 110 distancing from the second object 112, aspecific action, such as enlarging of an associated GUI element, istriggered. In a further embodiment the initiation and/or deactivation ofthe multi-object hovering input mode is further affected by the distancebetween the objects. For instance, a further condition for entering step304 may be the close proximity between the objects. In one embodimentthe distance between the objects is checked in connection with step 302of FIG. 3. The input mode could be set to be activated only when atleast two fingers are close enough to each other. Furthermore, the inputmode could be de-activated 318 when the objects 110, 112 are separatedfar enough from each other.

In one embodiment, a user may configure various settings associated withthe multi-object hovering mode. For instance, the user may specify afunction associated with an object 110, 112 and/or hovering area 250,252 to be applied in the multi-object hovering mode. A further exampleis that the user may select either left-hand or right-hand buttonarrangement, and the functionality associated with the objects isadapted accordingly. In a further example, further associations areadded in accordance with user inputs. For instance, the user may link anaction to a further specific hovering input. Thus, the user may tailorthe functionality of the multi-object mode according to his or herpreferences.

In some embodiments the present features are applied in connection withuser interfaces providing 3D interaction, or sense of 3D interaction.For instance, various auto-stereoscopic screens may be applied. Theproximity detection system 120, 220 may be configured to determine thedistance from the surface 100 and select an action, such as a displayoutput, dependent on the distance. For instance, a GUI element isenlarged when the objects 110, 112 are receding from the surface 100.

In one embodiment the apparatus 10, 200 is configured to detect gesturesby one or more objects (separately or in combination) in the hoveringarea 140, 240 above the input surface to trigger the multi-objecthovering mode or to detect inputs during the multi-object hovering mode.Changes in the proximity sensing field may thus be monitored. A gestureis identified based on the detected changes. An action associated withthe identified gestures may then be performed. For instance, gesturebased inputs may be applied in connection with at least some of theoperations in FIGS. 3 to 6 d. A very simple example is that in responseto detecting a gesture similar to movement of a finger whendouble-clicking a mouse button, a GUI element is selected and a furtheraction is performed for the selected GUI element, e.g. a new applicationwindow with contents of a selected URL is displayed. Another example isthat the apparatus 10, 200 is configured to detect movement of an objectto form a circle and thus move to a subsequent or preceding view.

As already indicated, inputs during the multi-object hovering mode maybe associated with touch and/or hovering inputs by the objects 110, 112.For instance, an input on the touch screen 210 by the second object 112,such as a right-side finger, may be associated with a tap selectionfunction, whereas non-touch hovering inputs by the first object 110,such as the left-side finger may be simultaneously monitored.

In one embodiment the apparatus 10, 200 is configured to control userinterface action on the basis of further properties associated withmovement of a single object or a combination of objects 110, 112 duringthe multi-object hovering mode. For instance, the apparatus 10, 200 maycomprise a pressure sensor connected to the controller 130, 230, and UIoperations may be controlled in accordance with the detected pressure.In another example the apparatus is configured to control actions on thebasis of time between consecutive inputs, time of touch to the touchscreen 210, speed of the movement of the object 110, 112 in the hoveringarea 140, 240, etc. For instance, two or more actions may be selectableon the basis of speed of the object relative to the touch screen 210,e.g., fast and slow touch inputs could be separated. Such furtherspecific inputs can be detected during the multi-object hovering mode,for instance as additional checks or further checks in connection withat least one of operations 308 to 314 of FIG. 3.

The above-illustrated features may be applied for different applicationsand applications modes. Some examples of applications for whichparticular advantages are available include browser applications,applications utilizing maps, media content related applications andservices, applications utilizing augmented reality interactions, and thelike.

Although some examples have been illustrated above in connection with asmall display, the present hovering related features may be applied inconnection with large displays and surfaces. Further, the proximitydetection system 120, 220 may be arranged to detect fingers of multiplehands. For instance, the system may support hovering by both hands.Further, the system may support detection of hovering by two or moreusers interacting with different areas of a large display.

FIG. 8 shows a block diagram of the structure of an electronic device800 according to an example embodiment. The electronic device maycomprise the apparatus 10 or 200. Although one embodiment of theelectronic device 800 is illustrated and will be hereinafter describedfor purposes of example, other types of electronic devices, such as, butnot limited to, portable digital assistants (PDAs), pagers, mobilecomputers, desktop computers, laptop computers, media players,televisions, gaming devices, cameras, video recorders, positioningdevices, electronic books, wearable devices, projector devices, andother types of electronic systems, may employ the present embodiments.Furthermore, the apparatus of an example embodiment need not be theentire electronic device, but may be a component or group of componentsof the electronic device in other example embodiments.

A processor 802 is configured to execute instructions and to carry outoperations associated with the electronic device 800. The processor 802may comprise means, such as a digital signal processor device, amicroprocessor device, and circuitry, for performing various functionsincluding, for example, one or more of the functions described inconjunction with FIGS. 1 to 7. The processor 802 may control thereception and processing of input and output data between components ofthe electronic device 800 by using instructions retrieved from memory.The processor 802 can be implemented on a single-chip, multiple chips ormultiple electrical components. Some examples of architectures which canbe used for the processor 802 include dedicated or embedded processor,and ASIC.

The processor 802 may comprise functionality to operate one or morecomputer programs. Computer program code may be stored in anon-transitory or tangible memory 804. The at least one memory and thecomputer program code may be configured to, with the at least oneprocessor, cause the apparatus to perform at least one embodimentincluding, for example, one or more of the functions described inconjunction with FIGS. 1 to 7. Typically the processor 802 operatestogether with an operating system to execute computer code and produceand use data.

By way of example, the memory 804 may include non-volatile portion, suchas EEPROM, flash memory or the like, and a volatile portion, such as arandom access memory (RAM) including a cache area for temporary storageof data. The information could also reside on a removable storage mediumand loaded or installed onto the electronic device 800 when needed.

The electronic device 800 may comprise an antenna (or multiple antennae)in operable communication with a transceiver unit 806 comprising atransmitter and a receiver. The electronic device 800 may operate withone or more air interface standards and communication protocols. By wayof illustration, the electronic device 800 may operate in accordancewith any of a number of first, second, third and/or fourth-generationcommunication protocols or the like. For example, the electronic device800 may operate in accordance with wireline protocols, such as Ethernetand digital subscriber line (DSL), with second-generation (2 G) wirelesscommunication protocols, such as IS-136 (time division multiple access(TDMA)), Global System for Mobile communications (GSM), and IS-95 (codedivision multiple access (CDMA)), with third-generation (3 G) wirelesscommunication protocols, such as 3 G protocols by the Third GenerationPartnership Project (3 GPP), CDMA2000, wideband CDMA (WCDMA) and timedivision-synchronous CDMA (TD-SCDMA), or with fourth-generation (4 G)wireless communication protocols, wireless local area networkingprotocols, such as 802.11, short-range wireless protocols, such asBluetooth, and/or the like.

The user interface of the electronic device 800 may comprise an outputdevice 808, such as a speaker, one or more input devices 810, such as amicrophone, a keypad or one or more buttons or actuators, and a display812 for displaying information in two or more dimensions.

The input device 810 may include a touch sensing device configured toreceive input from a user's touch and to send this information to theprocessor 802. Such touch-sensing device may be configured to recognizealso the position and magnitude of touches on a touch sensitive surface.The touch sensing device may be based on sensing technologies including,but not limited to, capacitive sensing, resistive sensing, surfaceacoustic wave sensing, pressure sensing, inductive sensing, and opticalsensing. Furthermore, the touch sensing device may be based on singlepoint sensing or multipoint sensing. In one embodiment the input deviceis a touch screen, which is positioned in front of the display 812. Theinput device 810 may be configured to operate, in response to thedetection (302) of simultaneous proximity of a first object and a secondobject, in an input mode, in which a first action is associated with aninput by the first object and a second action is associated with aninput by the second object.

The display 812 could be of any type appropriate for the electronicdevice 800 in question, some examples include plasma display panels(PDP), liquid crystal display (LCD), light-emitting diode (LED), organiclight-emitting diode displays (OLED), projectors, holographic displaysand the like. Hovering could be arranged even above a surface on whichdisplay is projected. For instance, cameras could be used to detect theproximity and position of the fingers.

The electronic device 800 also comprises a proximity detection system814, such as the system 120, 220 illustrated earlier, operativelycoupled to the processor 802. The proximity detection system 814 isconfigured to detect when a finger, stylus or other pointing device isin close proximity to, but not in contact with, some component of thecomputer system including for example housing or I/O devices, such asthe touch screen.

The electronic device 800 may comprise also further units and elementsnot illustrated in FIG. 8, such as further interface devices, a battery,a media capturing element, such as a camera, video and/or audio module,and a user identity module.

Embodiments of the present invention may be implemented in software,hardware, application logic or a combination of software, hardware andapplication logic. In an example embodiment, the application logic,software or an instruction set is maintained on any one of variousconventional computer-readable media. In the context of this document, a“computer-readable medium” may be any media or means that can contain,store, communicate, propagate or transport the instructions for use byor in connection with an instruction execution system, apparatus, ordevice, such as a computer, with one example of a computer described anddepicted in FIG. 8. A computer-readable medium may comprise acomputer-readable storage medium that may be any media or means that cancontain or store the instructions for use by or in connection with aninstruction execution system, apparatus, or device, such as a computer.

If desired, at least some of the different functions discussed hereinmay be performed in a different order and/or concurrently with eachother. Furthermore, if desired, one or more of the above-describedfunctions may be optional or may be combined.

Although various aspects of embodiments of the invention are set out inthe independent claims, other aspects comprise other combinations offeatures from the described embodiments and/or the dependent claims withthe features of the independent claims, and not solely the combinationsexplicitly set out in the claims.

It is also noted herein that while the above describes exampleembodiments of the invention, these descriptions should not be viewed ina limiting sense. Rather, there are several variations and modificationswhich may be made without departing from the scope of the presentinvention as defined in the appended claims.

The invention claimed is:
 1. An apparatus, comprising: at least oneprocessor; and at least one memory including computer program code, theat least one memory and the computer program code configured to, withthe at least one processor, cause the apparatus at least to perform:receive an indication of a presence of one or more objects hovering inclose proximity to an input surface; in an instance in which only asingle object is indicated to be hovering in close proximity to theinput surface, initiate a single input mode in which a first action isperformed by the processor in response to subsequent touch input by thesingle object to the input surface; in an instance in which both firstand second objects are indicated to be hovering in close proximity tothe input surface, initiate a multi-input mode; and while in themulti-input mode, determine a relational position of the first andsecond objects by determining a positional relationship of the first andsecond objects in a direction extending outwardly from the input surfacein order to determine which one of the first and second objects providestouch input to the input surface and which one of the first and secondobjects remains hovering in close proximity to the input surface;perform a second action by the processor in response to touch input bythe first object, as determined by the relational position of the firstand second objects, to the input surface without touch input by thesecond object to the input surface; and perform a third action by theprocessor in response to touch input by the second object, as determinedby the relational position of the first and second objects, to the inputsurface without touch input by the first object to the input surface,wherein the second action is different from the first and third actionsand the third action is different from the first action.
 2. Theapparatus of claim 1, wherein the apparatus is configured to deactivatethe multi-input mode in response to receiving an indication of removalof at least one of the first object or the second object from closeproximity to the input surface.
 3. The apparatus of claim 2, wherein theapparatus is configured to control at least one of the activation ordeactivation of the multi-input mode on the basis of distance betweenthe objects.
 4. The apparatus of claim 1, wherein the apparatus isconfigured to: determine an operation in response to a touch input by atleast one of the first object or the second object further on the basisof the position of the at least one of the first object or the secondobject.
 5. The apparatus of claim 1, wherein the apparatus is configuredto, in response to receiving the touch input by the first object in themulti-input mode, perform a selection action for an information elementassociated with the touch input by the first object, and in response toreceiving the touch input by the second object, display a menu ofselectable items for an information element associated with the touchinput by the second object.
 6. The apparatus of claim 1, wherein theapparatus is configured to modify at least one association to agraphical user interface element to be displayed in response to theactivation of at the multi-input mode or an action during themulti-input mode.
 7. The apparatus of claim 1, wherein the apparatus isconfigured to: cause display of a pointer element in response to thereceiving the indication of the first object and the second objectsimultaneously in close proximity to the input surface, and causedisplay of the pointer element to move in relation to movement of atleast one of the first object or the second object.
 8. The apparatus ofclaim 1, wherein an operation is associated during the multi-input modewith the combination of the touch input by the first object and thetouch input by the second object.
 9. The apparatus of claim 1, whereinthe apparatus is a mobile communications device comprising a touchscreen and a proximity detection system with at least one proximitydetector.
 10. A method, comprising: receiving an indication of presenceof one or more objects hovering in close proximity to an input surface;in an instance in which only a single object is indicated to be hoveringin close proximity to the input surface, initiating a single-input modein which a first action is performed by a processor in response tosubsequent touch input by the single object to the input surface; in aninstance in which both first and second objects are indicated to behovering in close proximity to the input surface, initiating amulti-input mode; and while in the multi-input mode, determining arelational position of the first and second objects by determining apositional relationship of the first and second objects in a directionextending outwardly from the input surface in order to determine whichone of the first and second objects provides touch input to the inputsurface and which one of the first and second objects remains hoveringin close proximity to the input surface; performing a second action bythe processor in response to touch input by the first object, asdetermined by the relational position of the first and second objects,to the input surface without touch input by the second object to theinput surface; and performing a third action by the processor inresponse to touch input by the second object, as determined by therelational position of the first and second objects, to the inputsurface without touch input by the first object to the input surface,wherein the second action is different from the first and third actionsand the third action is different from the first action.
 11. The methodof claim 10, further comprising: deactivating the multi-input mode inresponse to receiving indication of removal of at least one of the firstobject or the second object from close proximity to the input surface.12. The method of claim 10, wherein, an operation is determined inresponse to a touch input by at least one of the first object or thesecond object further on the basis of the position of the at least oneof the first object or the second object.
 13. The method of claim 10,wherein at least one association to a graphical user interface elementto be displayed is modified in response to the activation of themulti-input mode or an action during the multi-input mode.
 14. Themethod of claim 10, further comprising: in response to receiving anindication of the touch input by the first object, performing aselection action for an information element associated with the touchinput by the first object, and in response to receiving an indication ofthe touch input by the second object in the multi-input mode, displayinga menu of selectable items for an information element associated withthe touch input by the second object.
 15. The method of claim 10,wherein at least one of the activation or deactivation of themulti-input mode is controlled on the basis of distance between theobjects.
 16. The method of claim 10, further comprising: causing displayof a pointer element in response to the indication of the presence ofthe first object and the second object simultaneously, and causingdisplay of the pointer element to move in relation to movement of atleast one of the first object or the second object.
 17. A computerprogram product comprising a non-transitory computer readable storagemedium comprising one or more sequences of one or more instructionswhich, when executed by one or more processors of an apparatus, causethe apparatus to at least perform: receive an indication of a presenceof one or more objects hovering in close proximity to an input surface;in an instance in which only a single object is indicated to be hoveringin close proximity to the input surface, initiate a single input mode inwhich a first action is performed in response to subsequent touch inputby the single object to the input surface; in an instance in which bothfirst and second objects are indicated to be hovering in close proximityto the input surface, initiate a multi-input mode; and while in themulti-input mode, determine a relational position of the first andsecond objects by determining a positional relationship of the first andsecond objects in a direction extending outwardly from the input surfacein order to determine which one of the first and second objects providestouch input to the input surface and which one of the first and secondobjects remains hovering in close proximity to the input surface;perform a second action in response to touch input by the first object,as determined by the relational position of the first and secondobjects, to the input surface without touch input by the second objectto the input surface; and perform a third action by the processor inresponse to touch input by the second object, as determined by therelational position of the first and second objects, to the inputsurface without touch input by the first object to the input surface,wherein the second action is different from the first and third actionsand the third action is different from the first action.
 18. Anapparatus, comprising: means for receiving an indication of presence ofone or more objects hovering in close proximity to an input surface; inan instance in which only a single object is indicated to be hovering inclose proximity to the input surface, means for initiating asingle-input mode in which a first action is performed in response tosubsequent touch input by the single object to the input surface; meansfor receiving an indication of simultaneous presence of the first objectand a second object hovering above and in close proximity to the inputsurface, and in an instance in which both first and second objects areindicated to be hovering in close proximity to the input surface, meansfor initiating a multi-input mode; and while in the multi-input mode,means for determining a relational position of the first and secondobjects by determining a positional relationship of the first and secondobjects in a direction extending outwardly from the input surface inorder to determine which one of the first and second objects providestouch input to the input surface and which one of the first and secondobjects remains hovering in close proximity to the input surface; meansfor performing a second action in response to touch input by the firstobject, as determined by the relational position of the first and secondobjects, to the input surface without touch input by the second objectto the input surface; and means for performing a third action inresponse to touch input by the second object, as determined by therelational position of the first and second objects, to the inputsurface without touch input by the first object to the input surface,wherein the second action is different from the first and third actionsand the third action is different from the first action.